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ABSTRACT 



The Physics Education Research Group at the University of 
Minnesota has developed an interview tool to investigate physics faculty 
views about the learning and teaching of problem solving. In the part of the 
interview dealing with grading, faculty members were asked to evaluate a set 
of five student solutions and explain their reasons for the grades that they 
assigned. Preliminary analysis on two of the five student solutions was done 
on. six physics faculty members from a large research university. The results 
indicate that faculty members hold conflicting beliefs when grading between 
valuing reasoning in student solutions and wanting to give students the 
benefit of the doubt. This paper illustrates the hypothesis that physics 
faculty hold conflicting values when grading, and describes how the research 
university faculty resolved their conflicts. (Author) 
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Instructors’ Ideas about Problem Solving - Grading 

Vince H. Kuo*, Kenneth Heller , Patricia Heller , Charles Henderson*, Edit Yerushalmi*^ 

*Physics Education Research Group, University of Minnesota, Minneapolis, MN 
^Weizmann Institute, Rehovot, Israel 

The Physics Education Research Group at the University of Minnesota has developed an interview 
tool to investigate physics faculty views about the learning and teaching of problem solving. In 
the part of the interview dealing with grading, faculty members were asked to evaluate a set of five 
student solutions and explain their reasons for the grades that they assigned. Preliminary analysis 
on two of the five student solutions was done on six physics faculty members from a large 
research university. The results seem to indicate that faculty members hold conflicting beliefs 
when grading: between valuing reasoning in student solutions and wanting to give students the 
benefit of the doubt. This paper will illustrate the hypothesis that physics faculty hold conflicting 
values when grading, and describe how the research university faculty resolved their conflicts. 



Introduction 

Many studies have found that introductory 
physics students come out of traditional 
physics classes able to solve test problems 
without mderstanding the physics concepts 
on which the problems are based'. This 
suggests that either the problems posed for 
students are inadequate, or the process by 
which they solve them is not the intended 
one. Curriculum developers have focused 
their efforts on two general ways of attacking 
this problem. Some emphasize directly 
building students’ conceptual knowledge^’^’"', 
while others emphasize developing student 
problem-solving skills^'". Although aspects 
of many of these curricula are reflected in 
faculty practices, seldom are they fully 
adopted. It is our belief that this lack of full 
adoption reflects a mismatch between the 
curriculum developers who design the 
instruction and the faculty for whom the 
instruction is designed. 

In order to understand the nature of this 
mismatch, we conducted interviews with 30 
physics faculty members to determine their 
beliefs and values. The purpose of this 
study is to generate a set of testable 
hypotheses on a common core of physics 
faculty values and beliefs about the teaching 
and learning of problem solving. The 
preliminary results from this study will serve 
as guidelines for further investigations on 
larger samples. 



This is the first in a series of three papers 
describing the study and its preliminary 
results, and will concentrate on the findings 
in regards to grading of student solutions. 
Grading is an important part of the 
curriculum since, in the student’s view, it 
communicates the expectations of the 
instructor. This paper will identify some 
values that physics faculty hold when they 
evaluate and interpret student performances 
from their problem solutions. The following 
two papers will discuss preliminary results 
from a detailed analysis of the six research 
university faculty members that we 
interviewed to determine: (1) their beliefs 
about student learning of problem solving in 
physics'^; and (2) relationships between the 
faculty’s beliefs about student learning and 
their expectations for their students' \ 

Interview 

The interview was based on the use of 
instructional artifacts'"' in physics 
instruction. All of the artifacts below dealt 
with the same physics problem given to the 
faculty to solve prior to the interview (see 
Figure 1). The artifacts used were: 

1. three instructor solutions that varied in 
the details of their explanation, physics 
approach, and presentation stmcture; 

2. five student solutions that were selected 
from actual final examination solutions 
at the University of Minnesota to 
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Figure 1 : Homework Problem. 



represent features of student practice, 
such as evidence of varying degrees of 
conceptual knowledge, knowledge 
organization, and organization of a 
decision-making process; and 
3. four problems types that represent a 
range of the types of problems used in 
introductory physics courses. 

Each individual interview with a faculty 
member lasted about VA hours and consisted 
of four parts. The first three parts of the 
interview, each dealing with one of the three 
artifacts, started with general questions 
about how and why the faculty used that 
type of artifact. The faculty interviewed was 
asked to compare each artifact to those he or 
she actually used. We concluded each part 
by asking the faculty to reflect on the 
problem-solving process, as represented in 
the artifacts.. During these three parts the 
interviewer noted each of the features of the 
problem-solving process that was mentioned 
on a separate index card, using the faculty 
member’s words. In the final part of the 
interview they were asked to sort these cards 
into categories of their choosing. They were 
then asked several questions about the 
categories, including their expectations 
about student learning of these problem- 
solving processes by the end of their course. 




Figure 2: Student Solution D (SSD). The 
boxed-off comments were written by the 
interviewers to indicate clearly identifiable 
mistakes so as to save time during the 
interview. 




The 30 faculty members interviewed were 
randomly chosen from a set of physics 
faculty located in Minnesota vdio had taught 
an introductory calculus -based physics 
course within the previous 5 years and could 
be visited and interviewed in a single day 
(107 possible; 35 contacted; 5 declined to 
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Values 


Faculty 


1 


2 


3 


4 


5 


6 


1. Instructors want to see reasoning in student solutions so they can 
know if a student really understands. 


V 




V 


V 


V 


V 


2. Instructors are reluctant to 
penalize a student who 
might be correct: 


• Burden of proof on the instructor; 

• View student solutions in best 
possible light. 






V 




V 


V 




V 


V 


V 


4 


V 


3. Conflict Resolution for 
SSE: 


• Insist on reasoning; 

• Compromise; 

• Give full credit. 


4 


N 

A 




7 








9 




9 






10 



Table 1: Conflicting Values and Resolutions - note that the V’s indicate the faculty holding that particular 
value, and the numbers are the grades that they assigned for SSE, Merged cells indicate a slight leaning of 
the faculty in that direction of the resolution. 



participate). The 30 participants were 
roughly divided equally among four types of 
higher education institutions: Community 
College (CC), State College (SC), Private 
College (PC), and Research University 
(RU). 

Grading 

The remainder of this paper will focus on 
the Student Solutions part of the interview. 
The faculty members were asked to grade 
five student solution artifacts on a ten-point 
scale. They were then asked to discuss the 
problem-solving knowledge or skills 
represented in each solution, and explain the 
reasons for their grading. 




Graph 1: Grade Comparison between SSD and 
SSE. 



The results presented in this paper are a 
comparison between two of the five student 
solutions. Student Solution D (SSD) and 
Student Solution E (SSE), shown in Figures 
2 and 3. SSD is a detailed solution, with 
explanations and reasons for the problem- 
solving process written in the solution. It 
has two mistakes that combine to yield the 
correct mmerical result. SSE also has the 
correct numerical answer, but has no 
explanations. Notice that SSE could have 
been using the same reasoning as SSD. 

A preliminary analysis compared the 
numerical grade that all thirty of the faculty 
members assigned to SSD and SSE. Graph 
1 indicates that the distribution was roughly 
evenly divided between those who gave 
SSD a higher grade (SSD > SSE) and those 
who gave SSE a higher grade (SSD < SSE), 
with a few that graded them both equally 
(SSD = SSE). The scoring of each solution 
differed vastly among individual faculty, 
especially on the grading of SSE. Only the 
community college faculty showed a 
noticeable bias toward one solution, 
preferring SSE. 

In order to understand these differences in 
grading, and to generate some hypotheses, 
we looked more closely at the interview 
transcripts of the six research university 
faculty, particularly the parts of the 
interviews that dealt with SSD and SSE. 
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During the interview five out of the six 
research university faculty expressed 
conflicting values when grading SSE (see 
Table 1). Although these five faculty 
members stated that they valued reasoning 
in the student solution, only one actually 
graded accordingly. The rest of the faculty 
were reluctant to penalize a student who 
might have the correct understanding. 
Faculty #6, while commenting on SSE, 
stated that, “there’s nothing really wrong 
with this. You’d have to question, though, 
whether or not he was thinking of the fact 
that energy is conserved as it’s on the string 
as well as when it’s flying. He didn’t show 
that he’s using the right principles (9 
points).” Judging fi*om this statement, and 
others like it, we can hypothesize that some 
of the faculty felt that the burden of proof 
fell on them to prove that the student dd not 
understand the physics. 

Other faculty members felt that they should 
always view the student solution in the best 
possible light. For example. Faculty #3 
commented, also about SSE, that, “the 
student fundamentally knows what’s going 
on (9 points).” The seemingly conflicting 
values of desiring explanation but wanting 
to give student the “benefit of the doubt” 
were resolved by the faculty mostly through 
assessing minor penalties (1 to 3 points out 
of 10) to SSE, and stating that they would 
include a comment conveying to the student 
that, “next time you should really show your 
reasoning.” 

Discussion 

The preliminary analysis of the grading part 
of the interview yielded a possible source of 
conflict within physics faculty teaching 
introductory physics. Based on this small 
sample, it appears that physics professors 
value seeing student reasoning in problem 
solutions, yet actually penalize this behavior 
in students. We intend to test this 
hypothesis by examining the values 
expressed in the other 24 faculty interviews. 
If the conflict between values and grading 



remains, a further study is also planned 
using tools suitable for a significantly larger 
sample. 
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